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ABSTRACT 

The University of North Carolina at Wilmington (UNCW) offers 
many of its courses through distance-education. Because UNCW offers courses 
through a distance-education format, these courses are eligible for 
accreditation review by the Southern Association for Colleges and School 
(SACS) . UNCW underwent its most recent SACS review during the 2001-2002 
academic year. The review of the distance-education courses for this review 
compared a distance-education course against the same course taught in a 
traditional classroom setting. The comparison for the SACS review did not 
control for several potential errors. From fall semester, 1998 through spring 
semester, 2002, there were 155 courses that were offered in a distance- 
education format. Of that number, 27 of the courses were also taught in a 
traditional classroom setting by the same instructor. Of that number, only 
two courses had enrollments in the distance-education course that were within 
10% of .the enrollments in the traditional education course. The purpose of 
this study, then, was to compare the means of the final grades of those 
courses (a management information systems class and a literature for children 
course) that were offered by the same instructor via a traditional education 
format and a distance-education format, controlling for textbook and semester 
offered. In both courses, no statistically significant difference was found. 
Six recommendations rose from this study: (1) all skill determinants, not 

just the final grades of the students, should be evaluated; (2) the quality 
of instruction should also be compared; (3) compare and evaluate the pre- 
existing skills and knowledge of the students registered in the two types of 
courses; (4) the students should be tracked into subsequent courses for a 
comparison of whether the knowledge gained from the current course, 
regardless of mode of instruction, provided a solid basis for the next 
course; (5) continue with the comparison of the other courses that were 
offered in both a traditional classroom setting and a distance-education 
setting during the same, semester when controlling for both the instructor and 
the textbook; and (6) present the results of this study to the chair of the 
SACS review for UNCW. Contains 29 references, and 2 tables and 3 figures of 
data. Appendixes contain data. (RS) 
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COMPARISON OF FINAL GRADES OF COURSES TAUGHT IN BOTH A TRADITIONAL 
CLASSROOM FORMAT AND A DISTANCE-EDUCATION FORMAT AT THE 
UNIVERSITY OF NORTH CAROLINA AT WILMINGTON 

Research Methodology 
by 

Laurie J. Patterson 
October, 2002 

The University of North Carolina at Wilmington (UNCW) offers many of its courses 
through distance-education. Because UNCW offers courses through a distance-education 
format, these courses are eligible for accreditation review by the Southern Association for 
Colleges and School (SACS). In fact, SACS specifically stated that an “institution must 
demonstrate that ... its distance-learning programs are effective and comply with all applicable 
criteria.” 

UNCW underwent its most recent SACS review during the 2001-2002 academic year. 
The review of the distance-education courses for this review compared a distance-education 
course against the same course taught in a traditional classroom setting. The comparison for the 
SACS review did not control for several potential errors: courses taught via the two modes were 
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not taught by the same instructor, did not use the same textbooks, and were not taught in the 
same semester. 

From fall semester, 1998 through spring semester, 2002, there were 155 courses that were 
offered in a distance-education format. Of that number, twenty-seven of the courses were also 
taught in a traditional classroom setting by the same instructor. Of that number, only two 
courses had enrollments in the distance-education course that were within 10% of the 
enrollments in the traditional education course. The purpose of this study, then, was to compare 
the means of the final grades of those courses that were offered by the same instructor via a 
traditional education format and a distance-education format, controlling for textbook and 
semester offered. 

The research question addressed for each course in this study was: “Is there a statistically 
significant difference between the means of the final grades of students when taught in a 
distance-education format as compared to those taught in a traditional classroom format while 
controlling for the same instructor and textbook?” The above hypothesis was tested at the level 
of .05 significance using a two-tailed independent samples t-test. Both courses were evaluated 
with one t-test value for each course. In both courses, no statistically significant difference was 
found. 

Six recommendations rose from this study. The first three recommendations were from 
the literature review. The first recommendation was that all skill determinants, not just the final 
grades of the students, be evaluated. The second recommendation suggested that the quality of 
instruction also be compared. The third recommendation was to compare and evaluate the pre- 
existing skills and knowledge of the students registered in the two types of courses. Another 
recommendation was that the students be tracked into subsequent courses for a comparison of 
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whether the knowledge gained from the current course, regardless of mode of instruction, 
provided a solid basis for the next course. The next recommendation was to continue with the 
comparison of the other courses that were offered in both a traditiqnal classroom setting and a 
distance-education setting during the same semester when controlling for both the instructor and 
the textbook. The last recommendation was to present the results of this study to the chair of the 
SACS review for UNCW. 
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Chapter One 
INTRODUCTION 

The University of North Carolina at Wilmington (UNCW) is a comprehensive general 
education institution located along the Atlantic sea coast and South Carolina border that 
primarily serves residents of the state. During fall semester 2002, 85% of the registered 
undergraduate students were in-state residents (UNCW, 2002) and 3.97% were first or second 
tier states surrounding North Carolina. 

UNCW has received high marks for its scholastic offerings. Scholastically, within the 
state's overall university system (UNC), UNCW ranks fourth for freshman SAT entrance scores. 
For the past five years, UNCW has led the UNC system for improvement in college entrance test 
scores. Since 1996 (when SAT scores were recentered), UNCW’s comprehensive SAT scores 
have risen from 1045 to 1092 (UNCW, 2001a). In 1999, ACT composite scores for this 
institution averaged 22 (The Princeton Review, 1999, p. 188). In the 2001 armual “America’s 
Best Colleges,” printed by U.S. News and World Report, the institution was ranked number eight 
among the best public regional universities in the South (Morse & Flanigan, 2000, p. 129). In 
addition, the 1997 publication of the ranking showed UNCW listed in the top ten as one of the 
best college values among southern regional universities, where the quality of the school is 
compared to the net cost of attending the school (Thompson & Morse, 1997, p. 111). 

Nature of the Problem 

Since 1997, when UNCW was first listed in U.S. News and World Report as one of the 
best public regional universities in the South, applications for admission have increased 1 7% 
(UNCW, 2001a). Partially in response to this increase in applications and partially in response 
to the need to remain competitive with other institutions, UNC’s General Administration (GA) 
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requested that each institution within the UNC system submit plans for expanded instruction 
using distance-learning (UNCW, 1996). Within UNCW, the chancellor made distance-learning a 
part of UNCW’ s technology initiative (UNCW, 1998). For several years, UNCW has offered 
courses online; the first online courses were offered during fall semester, 1998. 

Since UNCW now promotes distance-education as one of its modes of instruction, the 
university was required to provide evidence during a recent Southern Association of Colleges 
and Schools (SACS) accreditation review that their distance-education courses were comparable 
to the more traditional classroom versions of the same courses. Specifically, the SACS review 
states “an institution must demonstrate that ... its distance learning programs are effective and 
comply with all applicable criteria” (UNCW, 2001b). 

UNCW’s initial response to the SACS review provided a comparison between these two 
modes of delivery. Each course that was compared for the SACS review was evaluated solely 
through a comparison of each course’s mean final grade. The study acknowledged differences in 
class size and demographics as variable. It was stated in the SACS report that the differences in 
GPA were within the expected range of variability (UNCW, 2002, p. 384). The individual who 
prepared this section of the SACS report is not sure what tests were conducted other than 
determining the means of each course (L. Rogers, personal communication, October 4, 2002). 
The comparison also did not control for several potential errors: courses taught via the two 
modes that were not taught by the same instructor, did not use the same textbooks, and were not 
taught in the same semester. Therefore, the chair of UNCW’s SACS review committee 
requested a better review of the modes of delivery to create a better control for some of these 
variables (K.W. Spademan, personal communication, January 31, 2002). 
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Purpose of the Study 

The purpose of this ex post facto study was to compare the final grades of two courses 
that were offered by the same instructor via a traditional education format and a distance- 
education format. The first course, introduction to management information systems (PDS 31,3) 
was taught spring semester, 2000. The enrollment for the traditional classroom class was 44 
(three of whom withdrew by mid-semester and were not included in the final tally). The 
distance-education counterpart had an initial enrollment of 48 with one student withdrawing 
before mid-semester. This student’s information was also not included in the final tally. The 
second course, literature for children (Eng 380) was taught fall semester, 2001 . Its traditional 
classroom enrollment was 38, with an enrollment of 36 in its distance-education counterpart. 

Due to the fact that the courses offered extended across a range of subjects, each traditional 
education course format was compared with its distance-education counterpart. 

Significance to the Institution 

The UNC president mandated that all campuses of the UNC system offer distance- 
education courses, resulting in UNCW’s offering of many courses via a distance-education 
format. UNCW states that it offers a quality education to the residents of the southeast region of 
North Carolina. The courses offered via a distance-education format have not yet been 
accurately compared with their traditional classroom counterparts. A comparison of both 
courses would provide evidence to the institution regarding the success rate of their distance- 
education courses. 

This information would also provide the university with the statistics it needs for 
accreditation purposes. In addition, departments offering the courses would be able to assess the 
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effectiveness of their courses. The information could be utilized for planning purposes by 
departments, colleges, and the university. 

Relationship to Seminar 

This practicum was related to the Research Methodologies course. Specifically, this 
practicum involved a comparison of courses taught in a traditional classroom setting with the 
same courses taught in a distance-education format. The concepts and principles learned in the 
research methodologies course provided a basis for this comparison and aided in the evaluation 
of the data. 

Research Questions 

Two research questions were addressed in this study. The same question was addressed 
for each course. Specifically, the questions were: 

1 . “Is there a statistically significant difference between the means of the final grades of 
students in the introduction to management information systems course when taught in a 
distance-education format as compared to grades of students taught in a traditional 
classroom format while controlling for the same instructor and textbook?” 

2. “Is there a statistically significant difference between the means of the final grades of 
students in the literature for children course offered spring semester, 2002, when taught 
in a distance-education format as compared to grades of students taught in a traditional 
classroom format while controlling for the same instructor and textbook?” 

Research Hypotheses 

The purpose of this study was to determine how the independent variable or the method 
of instruction affected the dependent variable or the final grades of students in the two courses. 
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Specifically, two courses were compared and two research hypotheses were created. The same 
research hypothesis was listed for each course compared. 

1. “There will be no statistically significant difference between the means of the final grades 
of students in the introduction to management information systems course when taught in 
a distance-education format as compared to grades of students taught in a traditional 
classroom format while controlling for the same instructor and textbook?” 

2. “There will be no statistically significant difference between the means of the final grades 
of students in the literature for children course offered spring semester, 2002, when 
taught in a distance-education format as compared to grades of students taught in a 
traditional classroom format while controlling for the same instructor and textbook?” 

Definition of Terms 

Dependent Variables. In this ex post facto study, the dependent variable was the final 
grade a student received in a course that is being compared. 

Distance-education. Distance-education is defined as providing education or training to 
an off-campus site via audio, video, or computers. 

Distance-Education Students. Distance-education students refer to those students who 
are taking a course via a distance-education format. It does not distinguish between students 
who are located off-campus and taking the course or are located on-campus and taking the 
course. 

Independent Variables. In this ex post facto study, the independent variable was the 
method of instruction. 
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Chapter Two 

REVIEW OF THE LITERATURE 
Introduction 

Within the U.S., education and ways of sharing and using knowledge are important to 
society. It is also important that education meet the needs of society (Maehr and Braskamp, 
1986). U.S. Department of Education statistics from fall 1995 showed that many institutions 
were offering distance-education courses. Forty-four percent of 2- and 4-year institutions (such 
as UNCW) offered distance-education courses during the 1997-1998 academic year (U.S. 
Department of Education, 1999). The U.S. Department of Education estimated that in 1995, 
one-third of higher education institutions offered distance-education courses. During the same 
period, the Department of Education stated an additional 25% of post-secondary institutions 
were planning to offer a distance-education course within three years (U.S. Department of 
Education, 1998). 

It is believed that students do not receive a comparable education in a distance-educa.tion 
course because students do not have direct contact with an instructor (Chambers, 1993, p. 4). 
Most studies, however, that have evaluated the effectiveness of distance-education have found 
that it is as effective as the traditional form of teaching (Siantz, Pugh, and Appelman, 1995, p. 6). 
Data, from a study conducted in 1996 (Meskill, Swan, and Frazer, 1997, p. 15), suggest that 
web-based multimedia collaboration is as successful when compared to a traditional classroom 
setting. 

Some of the literature indicates that a comparison between a traditional classroom course 
and a distance-education course is faulty. The design of this type of study fails to consider a 
multitude of variables (such as the socio-demographics of the learner, learning styles, or 
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psychological theories) that may influence student learning (Lockee, Moore, and Burton, 2001, 
p. 60). To have a clearer grasp of any differences between the two versions of the courses being 
offered, as many of the variables as possible, such as level of information and interaction 
between the two classes, must be kept to a minimum (Barron, 1985, p. 13). 

While much of the literature reports that there is no significant difference between these 
two settings that can be determined by statistical tests, this determination is interpreted to mean 
that distance-education is as effective as a traditional classroom setting (Lockee, Moore, and 
Burton, 2001, p. 62). The “no significant difference” result is inconclusive and does not mean 
that either mode of instruction is good (Lockee, Moore, and Burton, 2001, p. 62). 

In this type of study, the null hypothesis is tested as the expected outcome. This means 
the null hypothesis is accepted as true. The final result, then, is that the researcher speculates 
about what happened to produce the end result and does not have hard data to support the 
speculation (Lockee, Burton, and Cross, 1999, p. 467). 

Similar Problems 

Traditionally, the determination of a course's success is based upon how well students 
demonstrate they have absorbed the content of the course' (National Institute of Standards and 
Technology, 1994). The demonstration is usually reported by test scores and grades. Studies, 
using this method that compare distance-education courses with the same course offered in a 
traditional setting tend to show that there is little statistical difference between the two courses 
(Gagne and Shepherd, 2001, p. 58). As noted earlier in this document, most of the data 
suggested that the offering of a course via distance-education would be as successful as a course 
taught in a traditional classroom setting (Jacobs, 1996; U.S. Department of Education, 1999; 
Barron, 1985; National Institute of Standards and Technology, 1994). 
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In one example, at Brigham Young University, a course in health and physical education 
was offered in both a traditional setting on campus and through an online setting (Davies and 
Mendenhall, 1998). Although the course was taught by different instructors, it was found that 
there was no significant difference between the traditional classroom and the online versions 
(Davies and Mendenhall, 1998, p. 16). In this course, students were given pre- and post-test 
fitness exams, so there was a better grasp of the initial skills of both sets of students (Davies and 
Mendenhall, 1998, p. 6). 

Research Methodologies and Designs 

In a media comparison study (a comparison of a distance-education course against 
another medium of instruction), it is assumed that each method of instruction used is unique 
(Lockee, Moore, Burton, 2001, p. 60). The research design uses the method of checking if an 
independent variable affects the dependent variable (Lockee, Moore, Burton, 2001 , p. 60). In 
these studies, the independent variable has been the method of instruction and student grades 
have been the dependent variables. Most of the studies reviewed kept the number of extraneous 
variables, such as the same textbook, tests and the same instructor, at a minimum to further 
control the possible differences between the two modes of instruction. The most frequently used 
assessment (a comparison between homework and exam grades) is one that pre-exists in the 
traditional classroom format (Ryan, 2000, p. 79). This assessment is the comparison between 
homework and exam grades. 

A variety of tests were conducted to compare these course assessments. The analysis of 
variance (ANOVA) is a test that compares the means of two samples and then tests where the 
differences between the two means is statistically significant. In a study comparing a distance- 
education and traditional classroom graduate accounting class, a one-way fixed effect ANOVA 
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was used to compare four performance measures (tests and homework assignments) (Gagne and 
Shepherd, 2001, p. 60). In a comparison conducted for the Fischler Graduate School of 
Education and Human Services, Nova Southeastern University, the ANOVA was used to 
compare the means of final grades of students in traditional classroom courses and students who 
took the same course via the Internet (Fredda, 2000, p. 2). 

In a comparison conducted for Indiana State University, the ANOVA test was expanded 
out to compare a variety of means for the two different modes of instruction. The ANOVA was 
used to compare the means of the two samples’ final grades for a semester and across multiple 
semesters in which the two modes of instruction for the course were used (Yaw and Gilman, 
1999, p. 8). 

To determine the means of the final grades for use by the ANOVA test, the letter grades 
were converted to numbers on a 4.0 scale, with “A” = 4.0, “B” = 3.0, and so on (Yaw and 
Gilman, 1999, p. 8). In one case, only the means of “successful” grades were compared. 
Successful grades were identified as A, B, C, and Pass or Satisfactory (MacFarland, 1998, p. 3). 

In addition to the ANOVA, some of the studies used a chi-square test. The chi-test 
measures how frequently data appears (Ravid, 1994, p. 221). In a report for Nova Southeastern 
University, the chi-square test compared the frequency that successful grades appeared. In this 
case, successful grades also used the above definition (McFarland, 1998, p. 3). In a study for the 
University of Oklahoma, a chi-square was used in conjunction with a t-test to determine if there 
were any socio-demographic conditions that may have influenced the course outcomes (Leasure, 
Davis, and Thievon, 2000, p. 3). 

While a variety of tests were conducted to compare these course assessments, the t-test 
was used most frequently. A t-test is used to determine if the differences between groups are 
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significantly different (Ravid, 1994, p. 176). Although differences are expected between groups, 
the t-test checks to see if the differences are caused by sampling errors or by chance (Ravid, 

1994, p. 176). For example, in a study conducted at Tyler Junior College in Tyler, Texas, the t- 
test was used to determine if there were pre-existing differences in students who registered for 
political science courses in the two modes of instruction (Glenn, 2001, p. 20). Using an 
independent samples t-test allowed researchers to compare the pretest scores between the two 
groups. 

Results of Similar Studies 

In a study conducted at the University of Okalahoma, two courses were compared over 
two different semesters. The final grades of students in the distance-education or online course 
were compared to those of students in the traditional classroom or lecture course. There was no 
significant difference between the final grades of students (Ryan, 2000, p. 81). 

At the Fischler Graduate School of School of Education and Human Services, Nova 
Southeastern University, many of the graduate courses are offered in an Internet-based format. 

A comparison of those final grades with the same course offered on-campus showed that the 
means of the grades for the Internet course were significantly greater in two of the four courses 
compared (Fredda, 2000, p. 5-6). The other two courses did not show any statistical difference 
between the Internet-based course and the on-campus course (Fredda, 2000, p. 3, 8). 

In a report for Indiana State University, comparisons for distance-education and 
traditional classroom course formats using means, standard deviations, and t-tests, showed there 
were no significant differences between the two formats (Yaw and Gilman, 1999, p. 6). The 
researchers, however, used a two-way ANOVA test on the grade points of the two course 
formats over three semesters. In two of the three semesters, the mean grade point of the students 
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in distance-education format was slightly higher than that of the students in the traditional 
classroom setting. There was, however, no significant difference between students’ final grades 
in the two course formats (Yaw and Gilman, 1999, p. 10). 
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Chapter Three 

METHODOLOGY AND PROCEDURES 
Methodology 

This study was conducted by using a research methodology. An ex post facto design 
compared the traditional classroom course against the distance-education course to determine if 
there was any statistically significant difference between the final grades of the two sets of 
courses. Research hypotheses were developed, the level of significance was determined, the null 
hypotheses were tested, and inferential statistics were used for the results. 

Procedures 
Data Collection 

There were five procedures used to complete this research practicum. The procedures are 
as follows. 

Step one was a literature review conducted using distance-education, traditional 
classroom, final grades, and other related terms as the search terms. This procedure was 
accomplished by using online database search engines such as ERIC (Educational Resources 
Information Center). 

Step two was the identification of two courses taught at UNCW that had a course section 
code of 260-274 (online with all instruction delivered via the Internet). This was accomplished 
through contacting UNCW’s registrar’s office and requesting a list of such courses. 

Step three was a review of sections for the same course offered in the same semester in a 
traditional classroom setting with the same instructor. Courses selected had similar enrollments 
between the traditional classroom version and the distance-education version. These courses 
were identified by course section codes of 001-259 (on-campus class and lab). This review 




22 



22 



entailed connecting to UNCW’s computer system, looking up the course that was offered online, 
and checking to see if the same course was taught by the instructor in a traditional classroom 
setting. 

Step four was collecting the various final grades from both sets of courses and plotting 
them in a spreadsheet format. The final grades of each of the samples were recorded in two 
spreadsheets to keep the information separate between the experimental and control groups. 
Because the grades at UNCW are represented on an ordinal grading scale of A-F with “A” 
equivalent to a numerical scale of 4 and “F” equivalent to a 0 (zero), the grades were converted 
to their numerical equivalent. Appendix A provides the numeric value of grades for UNCW 
courses. All grades were included, including those of students who failed the courses. The 
number of students who withdrew from the class with a “W” was noted as withdrawn. Students 
who withdrew from the class with a “WF” (Failure/Late Withdrawal) were treated as though they 
had received an “F. The means for each course were determined and a two-tailed independent 
samples t-test was used to compare the means of the experimental and control group samples. 

Step five was the publication of the data in a report. The report was submitted to the 
chair of the SACS review committee. 

Description of Population 

The population from which the sample was drawn for this study consisted of all students 
who had been registered in a traditional classroom course or in a distance-education course of 
two UNCW courses. These courses were “Introduction to Management Information Systems” 
and “Literature for Children.” These students were primarily from the state of North Carolina 
and were all undergraduate students. The information gathered from this study could be applied 
to all current and future on-campus and distance-education students who enroll in either the 
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introduction to management information systems or the literature for children course when both 
sections are taught by the same instructor. 

Description of Sample 

The population was divided into two groups: those who registered for the distance- 
education course and those who registered for the traditional classroom course. These two 
groups enrolled for the course during the same semester and used the same textbook and 
instructor. The two groups were defined as “experimental group” (distance-education setting) 
and “control group” (traditional classroom setting). The student determined the course in which 
s/he enrolled. The average size of each of the courses was 40 students. 

The student sample was determined by comparing distance-education courses taught 
since fall semester, 1998 with the same course taught in a traditional classroom setting during the 
same semester and by the same instructor. Courses that matched these criteria with similar 
enrollment sizes were chosen, resulting in two courses. 

Treatment: Control and Experimental Group 

The control group of the study was the traditional classroom setting for both courses 
reviewed. Despite the fact that both classes were housed in different departments, the courses 
were both handled in a similar manner. Both the traditional classroom settings of the 
introduction to management information systems and literature for children were taught in a 
classroom located on campus. 

The instruction format for the control group of introduction to management information 
systems was primarily a lecture format with the instructor standing at the front of the classroom 
and presenting the information. The instruction format for the literature for children control 
group was a discussion format with information provided by the instructor followed by 
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discussion between the students and the instructor. The management information systems course 
had in-class exams while the literature for children course had periodic assignments that were 
due throughout the semester. 

The distance-learning group was the experimental group. For the management 
information systems course, the course was taught via an asynchronous mode on the Internet. 

The students, for this section, were required to be on campus for an orientation to the course and 
to return to campus for periodic exams. For the literature for children course, the course was 
taught via a synchronous mode with interactive television. At no time were the students required 
to come to campus for instruction or exams. 

Data Analysis 

The final grades of each of the samples were recorded in spreadsheets to keep the 
information separate between the experimental and control groups. Grades at UNCW are 
represented on an A-F scale with “A” equivalent to a numeric scale of 4 and “F” equivalent to a 
0 (zero). The grades were converted to their numeric equivalent. All grades were included, 
including those of students who failed any of the courses. Grades from students who withdrew 
from the class with a “W” were ignored. Students who withdrew from the class with a “WF” 
(Failure/Late Withdrawal) were treated as though they had received an “F. The numeric 
equivalents were added together and the mean for each section was determined. The means of 
each course were then analyzed using statistical software with a two-tailed independent samples 
t-test. 

Null Hypothesis 

Two null hypotheses were created for this study. They were as follows. 
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1 . “There will be no statistically significant difference between the means of the final grades 
of students in the introduction to nianagement information systems course when taught in 
a distance-education format as compared to grades of students taught in a traditional 
classroom format while controlling for the same instructor and textbook?” 

2. “There will be no statistically significant difference between the means of the final grades 
of students in the literature for children course offered spring semester, 2002, when 
taught in a distance-education format as compared to grades of students taught in a 
traditional classroom format while controlling for the same instructor and textbook?” 

Alternative Hypothesis 

Two alternative hypotheses were created for this study. They were as follows. 

1 . “There will be a statistically significant difference between the means of the final grades 
of students in the introduction to management information systems course when taught in 
a distance-education format as compared to grades of students taught in a traditional 
classroom format while controlling for the same instructor and textbook?” 

2. “There will be a statistically significant difference between the means of the final grades 
of students in the literature for children course offered spring semester, 2002, when 
taught in a distance-education format as compared to grades of students taught in a 
traditional classroom format while controlling for the same instructor and textbook?” 

Level of Significance 

The above null hypothesis was tested at the level of .05 significance. This level of 
significance was chosen with some certainty that the results would fall within a 95% range of 
confidence. 
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Region of Rejection 

The region of rejection for the hypothesis described above was two-tailed. A two-tailed 
test was used since the research hypothesis is non-directional for effect of instructional method 
on grade outcomes. The 5% region of rejection area was equally divided between the two tails. 

Statistical Tests 

The arithmetic means of the final grades of each course was determined by adding all the 
corresponding letter grade values and dividing the result by the number of students who received 
a final grade. Once those values had been determined, the two-tailed independent samples t-test 
was used for comparing the means of the experimental and control group samples. The t-test 
was used because each of the courses was a small sample and was drawn from the same parent 
population. 

A statistical analysis software was utilized to compare the final grades or scores of the 
two groups. The means and standard deviations for each group were also determined. Results 
were kept separate for each of the different courses. 

Assumptions 

There were several assumptions required by this study. First, it was assumed that the 
instructors for both the traditional and distance-education courses provided consistent instruction 
for each of the two groups. Second, it was assumed that the instructors used the same criteria for 
grading in each of the two courses. Third, it was assumed that the information provided to 
students and the testing of those students was similar enough across the two methods of 
instruction to have been only a minor influence on the outcome. Fourth, it was assumed that all 
of the students registered in both sets of courses had an equal chance to pass the course. Fifth, it 
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was assumed that each group entered the course with the prerequisite skills needed to complete 
the course. 

Limitations 

The results were limited to the University of North Carolina at Wilmington. Within the 
two groups, students determined which mode of instruction they would take. The two 
populations were self-selected during the registration process. Other independent variables, such 
as pre-existing differences among the students and the times at which the courses were taken, 
could not be controlled. The study, then, is limited by circumstances that may be unique to the 
UNCW setting and the needs of its students and can only be generalized to students enrolled in 
either the introduction to management information systems or the literature for children course. 
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Chapter Four 
RESULTS 

The purpose of this study was to compare the University of North Carolina at 
Wilmington’s distance-education courses with the same course offered in a traditional classroom 
setting. The comparison was initiated to gather data for future accreditation visits to UNCW. 

The results for each procedure are discussed below. 

Procedure One: Literature Review 

For the first procedure, a literature review was conducted to collect information related to 
other students or research that compared the grades of students in a traditional classroom setting 
with those of students in a distance classroom setting. The literature was retrieved through 
several databases, including ERIC, Wilson Education Full Text, ProQuest Direct, and Digital 
Dissertations. Terms, such as distance-learning, distance-education, traditional classroom, final 
grades, comparison study, and other related terms were used as the search terms. 

The research confirmed that there were no statistically significant differences between the 
means of grades of students who take a distance-education course or the traditional classroom 
counterpart. There were instances of the distance-education course having a higher final mean, 
while there were instances of the traditional classroom having the higher mean grade, and still 
other examples where both modes of instruction had no significant difference between the final 
grades. 

Some of the research, however, did suggest that this type of study is invalid because it is 
not comparing or evaluating the socio-demographics of the learners or their learning styles 
(Lockee, Moore, and Burton, 2001, p. 60). The result of “no significant difference” is 
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inconclusive and does not evaluate whether the quality of instruction is good (Lockee, Moore, 
and Burton, 2001, p. 62) 

Procedure Two: Identification of Distance-education Courses 
The second procedure involved the identification of the courses that could be evaluated. 
Courses that matched the distance-education criteria were identified by their course section 
number. Courses with a section number of 260-274 (online with all instruction delivered via the 
Internet) or 845-859 (extension courses offered online) were selected. This was accomplished by 
contacting UNCW’s Office of the Registrar and requesting a list of courses with those specific 
section numbers. 

From fall semester, 1998 through spring semester, 2002, 155 courses were offered in a 
distance-education format. Courses that were offered during summer sessions were eliminated 
because the population of individuals who take summer classes is not necessarily representative 
of the j3opulation of students who take courses during the regular academic year. This 
elimination reduced the number of possible courses by ten. 

Procedure Three: Identification of Traditional Classroom Courses 
Procedure three was a review of other courses offered during the same semester as the 
above distance-education courses. The courses under review were all those courses that had a 
counterpart that was offered in a traditional classroom format. These courses were identified by 
course section numbers 001-199 (traditional instruction format) and 800-829 (extension with a 
traditional instruction format). In addition, only those sections that had distance-education 
sections and were taught by the same instructor were selected. This initial match reduced the 
number of courses from 145 to 27. 
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The number of courses was further reduced by comparing traditional courses offered 
during the day to distance-education courses offered during the day and traditional courses 
offered through extension with distance-education courses also offered during extension. It was 
believed that this reduction would increase the chance of drawing from a similar population. 

This reduced the number of courses’ to twelve. 

The courses were further reduced by checking for courses that had similar enrollments (to 
avoid comparing the means of grades in a large class to the means of grades in a small class). 
Courses with enrollments that were within 10% of each other were selected. This reduced the 
number of classes compared to two . 

Procedure Four: Calculating and Analyzing the Data 

Step four was the collection of the various final grades from both sets of courses and 
plotting them out in a spreadsheet format. The registrar at UNCW collected the grades that were 
recorded for each of the courses. Collecting the information from the registrar kept the data 
confidential, and no names were associated with the assigned grades. 

The final grades of each of the samples were recorded in an Excel spreadsheet to keep the 
information separate between the experimental and control groups. Each set of courses was 
given its own worksheet within the spreadsheet. Because the grades at UNCW are represented 
on an ordinal A-F scale with “A” equivalent to a numerical scale of 4 and “F” equivalent to a 0 
(zero), the grades were converted to their numerical equivalent. There were no incompletes (“I”) 
or unreported grades (“Z”) associated with either set of courses. Appendix A shows the numeric 
scale used by UNCW. All grades were included, including those of students who failed the 
courses. The students who withdrew from the class with a “W” were noted as withdrawn and 
were not included in the final calculation. Students who withdrew from the class with a “WF” 
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(Failure/Late Withdrawal) were treated as though they had received an “F.” The means for each 
course were determined and a two-tailed independent samples t-test was used to compare the 
means of the experimental and control group samples and to determine if there was a statistically 
significant difference in final grades between the distance-education course and the traditional 
classroom course in both sets of courses. 

For the introduction to management information systems course, a two-tailed region of 
rejection was used with a = 0.05 and a critical value of 2.013 at a degree of freedom of 46. The 
results (z = 1 .206679) showed there was no statistically significant difference in this course when 
both sections are taught during the same semester and controlled for both the instructor and 
textbook. Figure 1 shows the number of grades given for the introduction to management 
information systems course. The results were achieved using SPSS statistical software and the 
values confirmed manually. 
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Figure 1 . Grade spread for the introduction to management information systems course. 



The null hypothesis for this course stated that there would “be no statistically significant 
difference between the final grades of students in the introduction to management information 
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systems course when taught in a distance-education format as compared to those taught in a 
traditional classroom format while controlling for the same instructor and textbook.” The null 
hypothesis failed to be rejected for the introduction to management information systems course. 

For the literature for children course, a two-tailed region of rejection was used with a a = 
0.05 and a critical value of 1 .993 at a degree of freedom of 72. The results (z = .339497) showed 
there was no statistically significant difference in this course when both sections are taught 
during the same semester and controlled for both the instructor and textbook. Figure 2 shows the 
number of grades given for the literature for children course. The results were achieved using 
SPSS statistical software and the values confirmed manually. 
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a a- b+ b b- c+ c c- d+ d d- f w wf 
Figure 2. Grade spread for the literature for children course. 

The null hypothesis for this course stated that there would “be no statistically significant 
difference between the final grades of students in the literature for children course when taught 
in a distance-education format as compared to those taught in a traditional classroom format 
while controlling for the same instructor and textbook.” The null hypothesis failed to be rejected 
for the literature for children course. 
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Figure 3 shows the t-test scores for both the introduction to management information 
systems course and the literature for children course. Both independent t-test scores were below 
the critical values at a 0.05 level of significance. Appendix B shows the breakdown of grades for 
each course with the associated variance. A histogram of the mean values for the two courses 
can be found in Appendix C. 



□ actual critical value 
o expected critical value 



Introduction to Literature for Children 
Information Systems 

Figure 3. Independent t-test scores for each course evaluated at UNCW 

Table 1 shows each course with the individual means for both the traditional classroom format 
and the distance-education format, the degrees of freedom (df), t-values, and p-values. Students 
in the traditional classroom courses, as shown in Table 1, had a higher final grade point average 
in all courses. 
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Table 1. 

Individual Means for Traditional Classroom and Distance-Education Formats for Both Courses 



Source 


Means 


Degrees of 
Freedom 


t-values 


p-values 


Traditional- 

Classroom 

Course 


Distance- 

Education 

Course 


Introduction to 
Management 
Information 
Systems Course 


3.22 


2.94' 


46 


1.206679 


p>.05 


Literature for 
Children Course 


3.64 


3.61 


72 


.339497 


p>.05 
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Chapter Five 

DISCUSSION, CONCLUSIONS, IMPLICATIONS AND RECOMMENDATIONS 

Discussion 

To remain competitive with other institutions, UNCW has made distance-learning part of 
its technology initiative and an integral part of its curriculum. Because distance-education is 

t 

now promoted at UNCW, the university must collect data on these courses in addition to its 
traditional classroom courses for accreditation purposes. A recent SACS accreditation review 
required evidence that UNCW’s distance-education courses were comparable to the traditional 
classroom version of the same course (UNCW, 2001b). 

While UNCW’s initial response to the SACS review did provide for a comparison 
between these two modes of delivery, the comparison did not control for instructor, textbooks, or 
when the course was taught. This study provided that information to the university. The results 
showed SACS evaluators that the university has taken the time and effort to determine if there 
was a difference between the two formats of delivery. In addition, departments offering the 
courses can use this information to assess the effectiveness of their courses or for planning 
purposes. 

The results of this ex post facto study further supported the evidence found in the 
literature review that showed that the means of final grades of students in either the distance- 
education course or the traditional classroom course were not statistically significantly different. 
Chambers (1993) stated that there is a belief that students who take a course via distance- 
education do not receive an education comparable to those who take a course in the traditional 
classroom setting. Yet, studies by Siantz, et al. (1995), Meskill, et al. (1997), and Gagne and 
Shepherd (2001) have found that not to be the case. The results of the study also agreed with 
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MacFarland (1998) who found that when controlling for instructor there was no statistically 
significant difference between the two modes of instruction. Even if the critical value was 
increased to a 99% level of confidence the null hypothesis would fail to be rejected. 

To better compare these two types of teaching formats, as many as the variables as 
possible should be controlled or kept to a minimum (Barron, 1985, p. 13). This study controlled 
for instructor, textbook, and semester in which the course was offered; other variables, such as 
existing knowledge of the student, learning styles, or student age should also be considered. In 
this study, the age range for students in the traditional classroom setting of the introduction to 
management information systems was 20-63 with a mean age of 24.3. For the distance- 
education setting, the age range was 20-46 with a mean age of 23.5. For students taking the 
literature for children course, the traditional classroom had an age range of 19-56 with a mean of 
23.7 while the distance-education setting’s range was 19-64 with a mean of 25. With these small 
differences between the two formats of teaching, the ages were very similar and could be 
considered controlled. Table 2 shows the age demographic for each course. 

Table 2. 

Age Demographics for Both Courses 



Source 


Mode 


Mean 


Minimum Age 


Maximum Age 


Introduction to 
Management 
Information 
Systems Course 


Traditional- 

Classroom 

Course 


24.3 


20 


63 


Distance- 

Education 

Course 


23.5 


20 


46 


Literature for 
Children Course 


Traditional- 

Classroom 

Course 


23.7 


19 


56 


Distance- 

Education 

Course 


25 


19 


64 
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While some of the literature indicated that this type of comparison was faulty because it 
didn’t take into account those additional variables (Lockee, Moore, and Burton, 2001, p. 60), this 
study did directly control for instructor, textbook, and semester, and the disparity of ages within 
the various classes was held to a minimum. Recommendations for controlling for the other 
variables will be discussed later in this report. 

The results of these two comparisons showed that there is statistically no significant 
difference between the two modes of instruction when controlling for instructor, textbook, and 
when the course was taught. The result of no significant difference here means that the end 
results, student final grades, are not significantly different between the two modes of instruction. 
The result did not state that the course or instruction was good. This result was also supported in 
the literature (Lockee, et al., 2001, p. 62). 

While the comparison between the two means showed there was no significant statistical 
difference between the two modes of instruction for this course, the internal validity of the 
methodology for evaluating the students in either section can be questioned. In hindsight, the 
instructor may believe that s/he was equal in the evaluation because assignments and grades were 
the same in both sections. The validity can be questioned because this statement of equality is in 
hindsight and cannot be verified that, at the moment of grading or assigning grades, the 
instructor approached each student in both sections in the same manner. While this may also be 
said for the evaluation of a single section, it can be assumed that the grading may have occurred 
differently because of the instructor’s first-hand, intimate knowledge of the students in the 
traditional classroom setting. Without the constant personal contact that the traditional- 
classroom students receive, assessments of the distance-education students may have occurred 
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Internal validity may also have been influenced by the small sample size. The total 
number of students compared in the introduction to management information systems 48 and 72 
for the literature for children course. The interpretation of the results should be done carefully 
with the small sample. With a comparison over time, when the population of students who have 
taken the two courses becomes larger, the end results may be different. 

Conclusions 

Two conclusions resulted from this study. The conclusions were the result of the two 
research questions that asked if there was a statistically significant difference between the final 
grades of students in either the introduction to management information systems course or the 
literature for children course when taught in either a distance-education or a traditional 
classroom format when controlling for the same instructor and textbook. 

Results from the study showed there was no statistically significant difference between 
the two groups within the two sets of courses. • The independent samples t-test results failed to 
reject the null hypothesis at a 0.05 level with regard to the independent variable or method of 
instruction. 

Implications 

The results of this study led to two implications that should positively affect UNCW. 
First, the study compared the final grades of students registered in courses taught in a traditional 
classroom setting with those of a distance-education course. The information garnered in this 
study provides UNCW with evidence that two of its distance-education courses are as effective 
as the traditional classroom counterparts if final grades are compared. The second implication 
was that evidence is now available for future accreditation reviews by SACS. 
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This study provides information that could further improve education at UNCW. 
Additional comparisons could confirm that the two modes of instruction at UNCW are equally 
successful when reviewing final grades. This could encourage additional courses to be 
developed and offered, thus reaching more state residents (and potential distance-education 
students) who may want access to courses offered by UNCW. 

Recommendations 

As a result of this practicum, several recommendations can be made. The first 
recommendation arises from the literature review. All skill determinants, such as assignments, 
quizzes, and exams that are the same within the two methods of instruction, should be evaluated. 
This information would provide guidance on which method of instruction, if any, worked with 
any of the various skill determinants. In addition, it could also provide information regarding 
how students did in applying their new skills or knowledge. 

A second recommendation also arises from the literature review. Because the t-test does 
not assess whether the quality of instruction is better in one setting or the other, it is 
recommended that the quality of instruction between the two methods also be compared. 

A third recommendation arising from the literature review would be to compare and 
evaluate the pre-existing skills and knowledge of the students registered in the two types of 
courses. This comparison would determine if there was any statistically significant difference in 
those pre-existing skills of the students. 

The fourth recommendation is for students in courses that are prerequisites for additional 
courses within a program, such as the introduction to management information systems course 
that a further comparison be made of the students. This comparison would compare how the 
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students performed in subsequent classes that build on the skills or knowledge gained in the 
introductory course. 

The fifth recommendation would be to continue with additional comparison of other 
courses that are offered in both a traditional classroom setting and a distance-education setting 
during the same semester when controlling for both the instructor and the textbook. There are 
twenty-five courses that have already been offered that could be compared. In addition, as 
UNCW continues to offer courses in both formats, future courses should be compared in this 
manner as well. This continued comparison would provide information for future SACS 
accreditation studies of UNCW. Finally, building upon the need to provide information for 
accreditation purposes, the results of this study were given to the chair of the SACS review for 
UNCW. 
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Appendix A 

UNCW Numeric Grade Scale 



Grade 


Numeric Value 


■ 


A 


4.00 




A- 


3.67 




B+ 


3.33 




B 


3.00 




B- 


2.67 




C+ 


2.33 




c 


2.00 




c- 


1.67 




D+ 


1.33 




D 


1.00 




D- 


0.67 




F 


0.00 




I 


0.00 


Incomplete 


W 


0.00 


Withdrawn 


WF 


0.00 


Withdrawal Failing 


Z 


0.00 


Unreported Grades 




4 



o 
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Appendix B 
Grade Distribution 



Introduction to Management Information Systems 
Section 004 Grade Total Received 


Section 260 


Grade 


Total Received 


control 


a 


22 


experimental 


a 


16 




a- 






a- 






b+ 






b+ 






b 


11 




b 


20 




b- 






b- 






c+ 






c+ 






c 


5 




c 


6 




c- 






c- 






d+ 






d+ 






d 


1 




d 


2 




d- 






d- 






f 


2 




f 


3 




w 


3 




w 


1 




wf 






wf 








ri|=41 






ri2=47 


average 




3.219512195 


average 




2.936170213 


variance 




1.175609756 

1 


variance 




1.234967623 


Literature for Children 










Section 001 


Grade 


Total Received 


Section 260 


Grade 


Total Received 


control 


a 


12 


experimental 


a 


11 




a- 


15 




a- 


14 




b+ 


9 




b+ 


8 




b 






b 


2 




b- 


2 




b- 






c+ 






c+ 






c 






c 


1 




c- 






c- 






d+ 






d+ 




I 


d 






d 






d- 






d- 
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